Evaluating the Divergent Auto-Encoder (DIVA) as a Machine Learning Algorithm

The divergent auto-encoder (Kurtz, 2007) offers an alternative to the multi-layer perceptron (MLP) for classification learning via back-propagation. The artificial neural network classifies based on its success reconstructing the input features (from shared, reduced dimensionality recodings) in terms of a generative model of each category. Successful simulations of rapid human learning of elemental, non-linearly separable category structures suggest potential in machine learning. In a series of simulation studies using benchmarks problems from the UCI database, the divergent autoencoder showed learning and generalization performance comparable to state-of-the-art algorithms with several major advantages: no evidence of overfitting, low sensitivity to parameter settings, and fast runtimes. Discussion focuses on three issues: (1) for what types of problems is the divergent autoencoder better or worse than leading algorithms; (2) comparison with MLP as the default architecture for classification learning with artificial neural networks; (3) comparison with other (Bayesian) generative methods for classification learning.